1. Field of the Invention
The present invention relates to an image processing apparatus, an image processing method, and an image processing program to detect an object (an image portion) that has appeared in an image and stopped there or that has disappeared from the image with the elapse of time.
2. Description of the Related Art
There has conventionally been suggested an image processing apparatus to detect an object that has appeared in a monitored area and stopped there or that has disappeared from the monitored area (hereinafter referred to as an unmoving object), by image processing based on video data obtained through a camera.
For example, Patent Document 1 (Japanese Patent No. 2913882) suggests a method and apparatus of detecting an obstacle or a dropped object on a road. In this known art, appearance and disappearance of an object on a road are determined and detected on the basis of changes of a plus/minus sign of a difference between a sequentially-updated reference background image and a captured image.
Various methods can be used to update a reference background image, one of them being exponential smoothing. In the exponential smoothing, a background image Bt is updated in the manner indicated by expression (1) shown in FIG. 17. Herein, the background image Bt is obtained at time t, an input image is Ct, and a smoothing constant is α (0≦α≦1).
In the exponential smoothing, an effect of an input image can be mitigated by increasing the smoothing constant α. Also, the effect can be mitigated when a moving object exists in the input image. However, the effect becomes significant if many moving objects exist in the input image.
In order to overcome this problem, a weighted mixture of normal distributions can be used to update a reference background image. In the method using a weighted mixture of normal distributions, a temporal change in luminance of each pixel is detected. The details are described below.
A probability P(Xt) of a luminance Xt of a present (time t) pixel can be typically expressed by expression (2) shown in FIG. 17. A probability density function η in expression (2) can be expressed by expression (3) shown in FIG. 17. At this time, a covariance matrix Σk,t is assumed to be expressed by expression (4) shown in FIG. 17.
In the method using a weighted mixture of normal distributions, whether the luminance of each pixel belongs to any of k (k is a positive integer) normal distributions is determined. For example, when luminance data of each pixel is 8-bit image data, four luminance normal distributions shown in FIG. 18A are provided, and whether the luminance of each pixel belongs to which of the four luminance normal distributions is determined.
For example, whether a luminance Xt of a pixel is within the range of a mean value μk±2σk of the luminance normal distribution is determined. If the luminance Xt is within the range, the luminance Xt is determined to belong to the luminance normal distribution. Otherwise, the luminance Xt is determined not to belong to the luminance normal distribution. If the luminance Xt does not belong to any of the luminance normal distributions, a mean value μ of the luminance normal distribution of the smallest weight (described below) among the k luminance normal distributions is replaced by the luminance Xt of the pixel at that time.
Then, the weights ωk,t of the respective luminance normal distributions are updated for each pixel so that the weight of the luminance normal distribution to which the luminance Xt of the pixel belong becomes large and the weights of the other luminance normal distributions become small. More specifically, the weight ωk,t of the luminance normal distribution to which the luminance Xt of the pixel belong is updated in accordance with expression (5) shown in FIG. 17, whereas the weights ωk,t of the other luminance normal distributions are updated in accordance with expression (6) shown in FIG. 17. In expressions (5) and (6), α is a speed of updating the weight (0≦α≦1).
The mean values μt and the variances σ of the respective luminance normal distributions are updated based on expressions (7), (8), and (9) shown in FIG. 17.
In this way, information about the respective weights ωk,t of the plurality of luminance normal distributions is updated for each pixel, as shown in FIG. 18B. The mean value of the luminance normal distribution of the largest weight obtained in this method basically indicates the luminance of a pixel in a still image portion except a moving object.
Therefore, by monitoring a change in the mean value of a luminance normal distribution of the largest weight, a state where an object has moved and disappeared and a state where an object has appeared and stopped can be detected in units of pixels without being affected by a moving object. Further, by positionally combining detection results of the respective pixels, an unmoving object can be detected as a block of an image portion.